Range-Efficient Counting of Distinct Elements in a Massive Data

Efficient one-pass estimation of F0, the number of distinct elements in a data stream, is a fundamental problem arising in various contexts in databases and networking. We consider range-efficient estimation of F0: estimation of the number of distinct elements in a data stream where each element of the stream is not just a single integer, but an interval of integers. We present a randomized algorithm which yields an ( , δ)-approximation of F0, with the following time and space complexities (n is the size of the universe of the items): (1)The amortized processing time per interval is O(log 1 δ log n ). (2)The workspace used is O( 1 2 log 1 δ log n) bits. Our algorithm improves upon a previous algorithm due to Bar-Yossef, Kumar and Sivakumar [BYKS02], which requires O( 1 5 log 1 δ log n) processing time per item. This algorithm can also be used to compute the max-dominance norm of a stream of multiple signals, and significantly improves upon the previous best time and space bounds due to Cormode and Muthukrishnan [CM03]. This algorithm also provides an efficient solution to the distinct summation problem, which arises during data aggregation in sensor networks [NGSA04, CLKB04].